Skip to content

[DPG] tpcSkimsTableCreator: bugfix reserve tables size#16739

Open
lubynets wants to merge 4 commits into
AliceO2Group:masterfrom
lubynets:tpc-skims-reserve
Open

[DPG] tpcSkimsTableCreator: bugfix reserve tables size#16739
lubynets wants to merge 4 commits into
AliceO2Group:masterfrom
lubynets:tpc-skims-reserve

Conversation

@lubynets

@lubynets lubynets commented Jun 19, 2026

Copy link
Copy Markdown
Contributor
  • In the tpcSkimsTableCreator there are two nested loops: collisions and tracks (V0s, cascades). Currently the reservation of the output table size is done inside the collision loop, while it should be done once per dataframe. It brings to underreservation of the size, which had no fatal effect before (although performance could worsen), but after these PRs, DPL Analysis: use unsafe cursor when reserving AliceO2#15450 and AOD Writer: correct reserve for cpvClusters AliceO2#15512 it brings to a fatal error. This PR fixes this logical error by bringing table reservation outside the collision loop;
  • Currently the size to be reserved was equal to the number of input tracks (or $2N_{\mathrm{V0}} + N_{\mathrm{casc}}$), which is extremely conservative, since only small part of the input is propagated to the output. This PR introduces configurables, which define the ratio of the expected output table size to that of the input table. The default values are chosen to be by factor of ~5 larger than the "usual" ratio with "common" selection configuration;
  • Added QA histograms, where the dataframe-wise distribution of the output/input table size ratios are stored, so one can tune the mentioned configurables if they are too conservative (if they are too aggressive the code will crash).

Tagging @amaringarcia

@github-actions github-actions Bot added the dpg label Jun 19, 2026
@github-actions github-actions Bot changed the title tpcSkimsTableCreator: bugfix reserve tables size [DPG] tpcSkimsTableCreator: bugfix reserve tables size Jun 19, 2026
@github-actions

github-actions Bot commented Jun 19, 2026

Copy link
Copy Markdown

O2 linter results: ❌ 1 errors, ⚠️ 0 warnings, 🔕 0 disabled

@alibuild

alibuild commented Jun 21, 2026

Copy link
Copy Markdown
Collaborator

Error while checking build/O2Physics/o2 for ffa864c at 2026-06-21 07:23:

## sw/BUILD/O2Physics-latest/log
2026-06-21T07:23:24.134+0200 [14217:140212047093376] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.134+0200 [14217:140212047093376] [executioncontext.cpp:739] [ERROR] Failed to download output blob e0b27c720f43013ad65c9c02d00b589f50ba1ebc782abaf3287297698a31bd0c: 14: Socket closed
2026-06-21T07:23:24.133+0200 [14244:139848482016896] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.134+0200 [14244:139848482016896] [executioncontext.cpp:739] [ERROR] Failed to download output blob 7277805acb749505b05413e7a8e26ed483de7e833745213f29644e752b66b979: 14: Socket closed
2026-06-21T07:23:24.133+0200 [13762:140666125524608] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.136+0200 [13762:140666125524608] [executioncontext.cpp:739] [ERROR] Failed to download output blob 3a3129a1381eda38f32af588c283a65edbc0f230d2a4fb7543997f7080360826: 14: Socket closed
2026-06-21T07:23:24.133+0200 [14153:140409950871168] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.136+0200 [14153:140409950871168] [executioncontext.cpp:739] [ERROR] Failed to download output blob 7532b1b1f8c54f0e4aecca738734bbaf404f5f9cff0af6b14b94a556249f74f3: 14: Socket closed
2026-06-21T07:23:24.133+0200 [14046:139886292381312] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.136+0200 [14046:139886292381312] [executioncontext.cpp:739] [ERROR] Failed to download output blob b68bf2f233eb06f42882fb0e04b28189c3068e12917e866eeb8bab29014ec85c: 14: Socket closed
2026-06-21T07:23:24.133+0200 [14090:140409084665472] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.136+0200 [14090:140409084665472] [executioncontext.cpp:739] [ERROR] Failed to download output blob 2e8b2d9345ed7b0f40ee1927526c0526d640748fa4c3c5e4f7a0ca164dc37f4a: 14: Socket closed
2026-06-21T07:23:24.133+0200 [14115:139766676414080] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.137+0200 [14115:139766676414080] [executioncontext.cpp:739] [ERROR] Failed to download output blob b7c93f7e29172ece00a9fb8ba0dee2e81dd5e7a7ee52b1fec11219d46a0850ac: 14: Socket closed
2026-06-21T07:23:24.133+0200 [14043:140264812265088] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.137+0200 [14043:140264812265088] [executioncontext.cpp:739] [ERROR] Failed to download output blob 43e7fadcb5640093eb86cc7c40bb587d34d18eb787c4571e9887d11dae5b6e1d: 14: Socket closed
2026-06-21T07:23:24.134+0200 [14180:140135823494784] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.138+0200 [14180:140135823494784] [executioncontext.cpp:739] [ERROR] Failed to download output blob 9da05281615529da03800723d71951848a09d437861bcef66a1c7398d63f6f44: 14: Socket closed
2026-06-21T07:23:24.133+0200 [13936:139810228383360] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.137+0200 [13936:139810228383360] [executioncontext.cpp:739] [ERROR] Failed to download output blob 031a704f4648b6eec3669a83e1e67e889daf08197acbde18274ccb8c48e44002: 14: Socket closed
2026-06-21T07:23:24.133+0200 [13562:140036516068992] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.138+0200 [13562:140036516068992] [executioncontext.cpp:739] [ERROR] Failed to download output blob 42d7767658c78f589ec5a23fc59323108a249ec0577f9051e861ace7febba854: 14: Socket closed
2026-06-21T07:23:24.133+0200 [13756:140545744424576] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.139+0200 [13756:140545744424576] [executioncontext.cpp:739] [ERROR] Failed to download output blob 29d36035694a1da40c761252102bcea4c22bcfbbeb3cb4f597e8bd7bfafb9229: 14: Socket closed
2026-06-21T07:23:24.134+0200 [13761:140180476759680] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.138+0200 [13761:140180476759680] [executioncontext.cpp:739] [ERROR] Failed to download output blob e506c14b34bdaa6e555fbfedd86f6e73317dba422f99d6c08eb10ec5cb94f391: 14: Socket closed
2026-06-21T07:23:24.133+0200 [13935:140225768095360] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.137+0200 [13935:140225768095360] [executioncontext.cpp:739] [ERROR] Failed to download output blob 7515fa71ee9724c5081194b390e7dbf5df7b73ed959b111621fb5de20a9db980: 14: Socket closed
2026-06-21T07:23:24.134+0200 [13759:140042790456960] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.138+0200 [13759:140042790456960] [executioncontext.cpp:739] [ERROR] Failed to download output blob 9276db2c6aa56d047053fc17fe0feb781a3cb88791ee5fd09dd14557423bc39b: 14: Socket closed
2026-06-21T07:23:24.133+0200 [13991:140205469924992] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.138+0200 [13991:140205469924992] [executioncontext.cpp:739] [ERROR] Failed to download output blob b9bca48449e948e1630848e12fd2c476181eb0d2ec41cbbf9629ff4ecfc4b330: 14: Socket closed
2026-06-21T07:23:24.133+0200 [13760:140263356071552] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.140+0200 [13760:140263356071552] [executioncontext.cpp:739] [ERROR] Failed to download output blob db4faa67d5bf500189951a1e3c921fd7867719d8805bdfe76d62142b816ac9c8: 14: Socket closed
2026-06-21T07:23:24.133+0200 [13757:140423859756672] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.140+0200 [13757:140423859756672] [executioncontext.cpp:739] [ERROR] Failed to download output blob b5259a9085d46c60928146454363ec6c4991fd17f58ea50477517ddc3e4d787f: 14: Socket closed
2026-06-21T07:23:24.133+0200 [13612:140663862296192] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.143+0200 [13612:140663862296192] [executioncontext.cpp:739] [ERROR] Failed to download output blob 446eb6a6918eafb3fe0a96324633124585a765765a3f962286d05a93f69e535b: 14: Socket closed
2026-06-21T07:23:24.134+0200 [13758:140000685031040] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.143+0200 [13758:140000685031040] [executioncontext.cpp:739] [ERROR] Failed to download output blob e26ce166c501092a239ae44a89b827c83cf82e115cfd424eee6b867ce142c054: 14: Socket closed
2026-06-21T07:23:24.133+0200 [13418:140549949722240] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.143+0200 [13418:140549949722240] [executioncontext.cpp:739] [ERROR] Failed to download output blob ef73263ef461bf8e7d0df902648b6a2a85704561fca6ccd6d474750d38cb78a5: 14: Socket closed
2026-06-21T07:23:24.133+0200 [13501:139812656684672] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.142+0200 [13501:139812656684672] [executioncontext.cpp:739] [ERROR] Failed to download output blob 8f9dfe6f334536db211ba79e946db355b6328c26b082a2f0b64de148766d4621: 14: Socket closed
2026-06-21T07:23:24.134+0200 [13348:139874757381760] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.144+0200 [13348:139874757381760] [executioncontext.cpp:739] [ERROR] Failed to download output blob 532938c46f15fed4a17d53f35c189f1e4671296a27d580ca5e0854f0d6817db3: 14: Socket closed
2026-06-21T07:23:24.133+0200 [13676:140450202177152] [buildboxcommon_grpcretrier.cpp:177] [ERROR] Retry limit (0) exceeded for "ByteStream.Read()", last gRPC error was [14: Socket closed]
2026-06-21T07:23:24.144+0200 [13676:140450202177152] [executioncontext.cpp:739] [ERROR] Failed to download output blob b814a349539e93e504902daa1e206af4c4c5136b4441fde86e7b1c58f024a38e: 14: Socket closed
[0 more errors; see full log]

Full log here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@alibuild alibuild Awaiting requested review from alibuild alibuild is a code owner

@chiarazampolli chiarazampolli Awaiting requested review from chiarazampolli chiarazampolli is a code owner

@alcaliva alcaliva Awaiting requested review from alcaliva alcaliva is a code owner

@catalinristea catalinristea Awaiting requested review from catalinristea catalinristea is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

dpg

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@lubynets @alibuild